Recent years have seen a widespread use of MPEG-2 (Moving Picture Expert Group-2) (Non patent Literature 1) and H.264 (Non patent Literature 2) as techniques for compressing and coding (hereinafter referred to as coding) moving picture data. In these image coding techniques, each of images (hereinafter referred to also as pictures) in a moving picture are subjected to motion estimation with reference to at least one picture that temporally precedes or succeeds the image. Furthermore, such coding involves coding of estimated motion information (including specifications of estimation-source pictures and motion vectors) and differences from the results of the estimation. In the coding, each picture is segmented into square or rectangle areas (hereinafter referred to blocks) each of which has a defined number of pixels and is used as a unit of coding.
FIG. 1 is a diagram showing general motion estimation.
A description is given of a method for estimating motion information performed in coding an image. In FIG. 1, 1001 denotes a current picture to be coded, 1002 denotes a block as a coding target (a current block to be coded), 1003 denotes an estimation-source picture, 1004 denotes a block co-located with a current block 1002 within the estimation-source picture 1003, 1005 denotes a search range, 1006 denotes a reference block, and 1007 denotes a motion vector.
Motion information is estimated in units of a block. First, the current block 1002 is determined, and then a picture that has already been coded and temporally precedes or succeeds the current block 1002 is determined as the estimation-source picture 1003. Next, within the estimation-source picture 1003, an area surrounding the block 1004 co-located with the current block 1002 is set as the search range 1005. Next, in the set search range 1005, blocks each having the same size as that of the current block 1002 is sequentially checked to find the block having the highest similarity with the current block 1002, and the block having the highest similarity is determined as the reference block 1006. Next, the relational position of the reference block 1006 with respect to the block 1004 co-located with the current block 1002 is shown using the motion vector 1007.
In FIG. 1, a hatched oval area in the picture 1001 in the right illustration shows an area of a subject (such as a car) captured in the picture 1001, etc. A hatched area in a picture 1003 in the left illustration shows, for example, an area of this subject in the picture 1003. As shown in the diagram, for example, the position of the hatched area in the left illustration may be different from the position in the right illustration due to, for example, movement of the car.
FIG. 2 is a diagram showing an example of a conventional image coding apparatus.
Various configurations have been disclosed as the configurations of image coding apparatuses which perform motion estimation (for example, Patent Literature 1). In FIG. 2, 2001 denotes a motion estimation unit, 2002 denotes an image memory, 2003 denotes a motion compensation unit, 2004 denotes an intra prediction unit, 2005 denotes a prediction scheme selecting unit, 2006 denotes a difference image generating unit, 2007 denotes an orthogonal transform and quantization unit, 2008 denotes a coding unit, 2009 denotes an inverse orthogonal transform and inverse quantization unit, 2010 denotes an image reconstructing unit, 2011 denotes a loop filter unit, 2012 denotes a block reading unit, 2013 denotes a similarity degree calculating unit, 2014 denotes an evaluating unit, and 2015 denotes a motion vector generating unit. Operations performed by an image coding apparatus 2x are described with reference to FIG. 2.
An input image is coded in units of a block. The current block (the current block 1002 (FIG. 1)) is input to the motion estimation unit 2001. The motion estimation unit 2001 sets a search range (search range 1005), based on the position information of the input current block, and sequentially reads images within the search range from the image memory 2002 and evaluates the read images to detect the block (reference block 1006) having the highest correlation with the current block. The image memory 2002 stores coded images. Next, the motion vector (motion vector 1007) indicating the relational position that is in the detected block and determined with respect to the current block is output to the motion compensation unit 2003.
The motion compensation unit 2003 reads the reference image indicated by the input motion vector from the image memory 2002 to obtain the reference image, and outputs the reference image as a prediction image for inter prediction to the prediction scheme selecting unit 2005.
In addition to this, the input image is input to the intra prediction unit 2004. The intra prediction unit 2004 generates an intra prediction image and an intra prediction information, and output them to the prediction scheme selecting unit 2005.
The prediction scheme selecting unit 2005 selects one of the intra prediction scheme and the inter prediction scheme, and outputs the prediction image for use in the selected one of the schemes to the difference image generating unit 2006.
The difference image generating unit 2006 generates a difference image between the input image and the prediction image for the selected one of the schemes output by the prediction scheme selecting unit 2005, and outputs the difference image to the orthogonal transform and quantization unit 2007.
The orthogonal transform and quantization unit 2007 outputs data obtained by performing orthogonal transform and quantization on the input difference image to each of the coding unit 2008 and the inverse orthogonal transform and inverse quantization unit 2009.
The coding unit 2008 performs variable length coding on the input data, and outputs the variable-length coded data as the coded image.
The inverse orthogonal transform and inverse quantization unit 2009 performs inverse quantization and inverse orthogonal transform on the data subjected to the orthogonal transform and quantization to generate a difference image, and outputs the difference image to the image reconstructing unit 2010.
The image reconstructing unit 2010 adds the prediction image generated by the prediction scheme selecting unit 2005 and the difference image generated by the inverse orthogonal transform and inverse quantization unit 2009 to generate a reconstructed image, and outputs the reconstructed image to the loop filter unit 2011.
The loop filter unit 2011 performs loop filtering on the reconstructed image, and stores the filtered reconstructed image to the image memory 2002.
The structure of the motion estimation unit 2001 is described.
The motion estimation unit 2001 (FIG. 2) includes a block reading unit 2012, a similarity degree calculating unit 2013, an evaluating unit 2014, and a motion vector generating unit 2015.
The block reading unit 2012 sets a search range (search range 1005) based on the position information of the current block, and sequentially reads the images within the set search range in the form of blocks from the image memory 2002. The read blocks are assumed to be candidate prediction images.
The block reading unit 2012 outputs, to the similarity degree calculating unit 2013, the image data of the candidate prediction images and the positions thereof within the picture.
The similarity degree calculating unit 2013 calculates the similarity degree between the current block and each of the image data of the candidate prediction images input by the block reading unit 2012, and outputs the similarity degree to the evaluating unit 2014.
The evaluating unit 2014 compares the calculated similarity degree of the block having the highest similarity degree and the calculated similarity degrees of the respective input candidate prediction image blocks within the search range. When the comparison of the similarity degrees are completed for all the blocks within the search range, the position of the block having the highest correlation within the search range is output to the motion vector generating unit 2015.
The motion vector generating unit 2015 generates a motion vector, based on the position information of the current block and the position information that is of the block having the highest correlation within the search range and input by the evaluating unit 2014, and outputs the motion vector to the motion compensation unit 2003.
FIG. 3 is a diagram showing a structure of a general moving picture decoding apparatus.
Next, how to decode a coded image is described. In the decoding, a prediction (see the earlier-mentioned prediction image) is generated (hereinafter, this generation processing is referred to as motion compensation), based on coded motion information and using the specified estimation-source picture (see the estimation-source picture 1003 in FIG. 1) and the motion vector (see the motion vector 1007). An image (see the block 1002 in FIG. 1) is reconstructed by adding the prediction and separately-coded difference information (see the earlier-mentioned difference image) indicating the difference from the prediction.
In FIG. 3, 3001 denotes a variable length decoding unit, 3002 denotes a motion compensation unit, 3003 denotes a motion vector calculating unit, 3004 denotes a reference image obtaining unit, 3005 denotes an image memory, 3006 denotes a prediction image generating unit, 3007 denotes an intra prediction unit, 3008 denotes a prediction scheme selecting unit, 3009 denotes an inverse quantization and inverse orthogonal transform unit, 3010 denotes an image reconstructing unit, and 3011 denotes a loop filter unit.
Operations performed by an image decoding apparatus are described with reference to FIG. 3.
A coded image input to the moving picture decoding apparatus 3x (image decoding apparatus) is input to the variable length decoding unit 3001 (as for the coded image, see the earlier description of the coded image output by the coding unit 2008 of the image coding apparatus 2x).
The variable length decoding unit 3001 performs variable length decoding. Next, the variable length decoding unit 3001 outputs information required to calculate a motion vector from among the results of the decoding to the motion compensation unit 3002, outputs information required for intra prediction to the intra prediction unit 3007, and outputs data required to generate a difference image to the inverse quantization and inverse orthogonal transform unit 3009.
The motion compensation unit 3002 performs inter prediction to generate a prediction image.
The motion compensation unit 3002 further includes a motion vector calculating unit 3003, a reference image obtaining unit 3004, and a prediction image generating unit 3006.
The information output from the variable length decoding unit 3001 and input to the motion compensation unit 3002 is next input to the motion vector calculating unit 3003.
The motion vector calculating unit 3003 calculates a motion vector, and outputs the motion vector to the reference image obtaining unit 3004.
The reference image obtaining unit 3004 obtains, as a reference image, a square or rectangle area indicated by the motion vector (see the reference block 1006 shown by the motion vector 1007 in FIG. 1) In a decoded image, and outputs the reference image to the prediction image generating unit 3006.
The prediction image generating unit 3006 generates a prediction image from the input reference image, and outputs the prediction image to the prediction scheme selecting unit 3008.
The intra prediction unit 3007 performs intra prediction using the information required for the intra prediction output by the variable length decoding unit 3001, and outputs the prediction image to the prediction scheme selecting unit 3008.
The prediction scheme selecting unit 3008 selects one of the image obtained using the inter prediction scheme and output by the motion compensation unit 3002 and the image obtained using the intra prediction scheme and output by the intra prediction unit 3007, and outputs, to the image reconstructing unit 3010, the image obtained, as the prediction image to be used, using the selected one of the prediction schemes.
The inverse quantization and inverse orthogonal transform unit 3009 performs inverse orthogonal transform and inverse quantization on the data output by the variable length decoding unit 3001 to generate a difference image, and outputs the difference image to the image reconstructing unit 3010.
The image reconstructing unit 3010 generates a reconstructed image by adding the prediction image output by the prediction scheme selecting unit 3008 and the difference image output by the inverse quantization and inverse orthogonal transform unit 3009, and outputs the generated reconstructed image to the loop filter unit 3011.
The loop filter unit 3011 performs loop filtering on the input image, and outputs the filtered image as a decoded image. In addition, the loop filter unit 3011 stores the output image into the image memory 3005 so that the image is used as a reference image in decoding of the following pictures.